Historical Background: Operant and Respondent Behavior

The last two chapters showed how behavior in a range of situations, from fixed-interval schedules, through the radial maze to conditioning experiments with rats and bees, appears to be guided by the animal’s representation of its world, together with principles of Bayesian inference. Studies of classical conditioning illustrate the way these inference principles allow the animal to incorporate into its representation something about the causal structure of the environment. In this chapter, I say more about the kinds of inference that seem to underlie both operant and classical conditioning and show how inference leads to action. The middle part of the chapter presents a view of learning as a circular process in which surprise and novelty cause an animal to update its representation of the situation, which in turn leads to new activity, hence to a new situation, perhaps more surprises, and so on in a spiral that usually converges on an adaptive pattern. The next chapter discusses several standard phenomena: conditioned reinforcement, conditioned suppression, avoidance and escape, set, and extinction. I begin with a brief account of the historical background to the contemporary study of operant learning. This account introduces some important empirical results and provides a context for the rather different approach to operant learning I propose later.